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SUMMARY  PAGE 


PROBLEM 


To  determine  if  auditoryj  visual  or  the  bimodal  approach  is  best  for 
the  detection  of  simple  (single  tone)  stimuli. 


FINDINGS 


Redundant  signals  in  two  modalities,  as  opposed  to  a  single  signal 
in  one  modality,  collectively  improved  sensitivity  and  reduced  uncertainty 
of  an  operator  regarding  a  choice  response,  thus  providing  a  reduction  in 
detection  threshold.  Detection  threshold  to  a  target  was  not  negatively 
affected  when  attention  was  divided  between  two  modalities  and  noise  was 
presented  in  both. 


APPLICATION 

The  finding  of  improved  detection  in  each  modality  of  the  bimodal 
condition  supports  the  advantage  of  a  bimodal  input  approach  in  tasks  which 
involve  bisensory  stimulation  such  as  sonar,  radar,  and  ECM. 
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ABSTRACT 


IWenty  men  were  presented  background  noise  and  target  stimuli  in  either 
the  visual  or  auditory  modality,  or  in  both  at  once.  Auditory  and  visual 
detection  thresholds  were  lowest  when  functionally  redundant  targets  were 
presented  simultaneously  in  both  modalities.  It  appeared  that  two  redundant 
signals  collectively  improved  sensitivity  and  reduced  uncertainty  regarding 
a  choice  response,  thus  enabling  a  reduction  in  detection  threshold. 
Detection  threshold  to  a  single  target  was  not  negatively  affected  when 
attention  was  divided  between  two  modalities  and  noise  was  presented  in 
both.  The  finding  of  improved  detection  in  each  modality  of  the  bimodal 
condition  is  consistent  with  the  coactivation  explanation  of  bimodal 
facilitation  and  supports  the  advantage  of  a  bimodal  input  approach  in  tasks 
such  as  sonar,  radar,  and  air  traffic  control. 


INTRODUCTION 


Many  investigators  have  found  that  response  speed  and/or  sensitivity 
are  enhanced  when  individuals  respond  to  two  sources  of  redundant 
information  rather  than  just  one.  This  "redundant  signals"  effect 
has  been  reported  in  numerous  one-modality  studies  in  which  two  identical 
stimuli  are  simultaneously  presented.  Most  commonly,  these  studies  involve 
visual  letter  detection. ^ >5-7  jn  addition,  some  studies  have  examined 
the  redundant  signals  effect  by  presenting  redundant  information 
simultaneously  via  two  modalities,  ,3,8-lZ  in  these  studies  the 
presentation  of  functionally  redundant  signals  in  two  modalities  resulted  in 
a  response  gain  (i. e. ,  decreased  reaction  time  or  detection  threshold)  over 
at  least  one  of  the  two  modalities;  however,  when  stimulus  information  was 
different  in  each  modality  or  required  separate  responses  or  multiple 
targets  were  used,  a  response  decrement  resulted. 


A  variety  of  studies  have  investigated  the  effects  of  presenting  target 
stimuli  bimodally,  and  found  that  a  response  to  a  redundant  signal  may  be 
more  sensitive  or  faster  than  a  response  to  either  single  source  of 
information.  1,3,13-15  Nickerson^  referred  to  this  effect  as 
"energy  summation, "  whereby  two  stimulus  energies  are  combined  in  such  a  way 
that  the  total  energy  is  equivalent  to  increasing  the  intensity  of  one 
stimulus  alone.  Miller  ,  Shaw^,  and  others  have  argued  that  rather 
than  independent  activations  within  each  channel,  information  from  two 
attended  modalities  is  integrated  and  results  in  "coactivation. "  They  have 
suggested  that  combined  activation  occurs  after  a  coding  stage  but  prior  to 
a  decision  stage,  and  builds  at  a  faster  rate  than  activation  from  either 
modality  alone,  thus  accounting  for  the  redundant  signals  effect. 


Perhaps  the  strongest  evidence  for  a  bimodal  redundant  signals  effect 
comes  from  studies  on  detectability  of  two  signals  in  the  threshold  region. 
Some  detection  studies  have  obtained  improved  performance  by  using 
short-duration  bimodal  stimulus  presentations,  >1°“! '  whereas  others 
have  obtained  improved  detection  in  vigilance  paradigms,  y  Of 

these  studies,  Colquhoun1J  attempted  to  simulate  actual  sonar  displays 
in  a  vigilance  study  of  single  and  dual-modality  performance.  He  found 
detection  to  be  best  when  auditory  and  visual  displays  were  monitored 
concurrently,  and  that  auditory  was  better  than  visual  detection  when 
comparing  single  modes.  Kobus,  et  al.  studied  detection  performance 
on  actual  sonar  equipment  using  a  complex  sonar  task.  They  had  subjects 
detect  and  classify  five  targets  masked  by  noise  in  one  or  both  modalities, 
and  did  not  direct  the  subjects'  attention.  They  found  no  difference 
between  the  bimodal  condition  and  the  better  single  modality.  Possible 
reasons  for  the  lack  of  bimodal  facilitation  are  the  large  number  of  targets 
used,  and  the  uncertainty  associated  with  where  the  target(s)  would  be 
presented. 

It  is  common  in  an  operational  setting  (i.e.,  sonar,  radar,  air 
traffic  control)  for  an  operator  to  have  to  deal  with  information  in  two 
modalities;  the  information  may  be  either  redundant  or  not,  and  presented  in 
various  noise  backgrounds.  Having  to  divide  his  attention,  whether  the 
signals  are  redundant  or  not,  the  amount  of  information,  and  the  noise  can 
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all  influence  his  performance.  But  the  effects  when  these  variables  impinge 
simultaneously  is  still  not  well  understood. 

The  purpose  of  this  study  was  to  examine  differences  between  unimodal 
and  bimodal  presentation  of  signal  information  on  detection  when  the  target 
information  was  presented  in  a  complex  task  such  as  sonar  with  continually 
updated  displays.  In  this  task  the  effect  of  target  redundancy  was  examined 
as  subjects  were  required  to  detect  and  classify  targets  when  their 
attention  was  either  focused  on  one  modalitiy  or  divided  between  two 
modal ities. 

It  was  hypothesized  that  detection  threshold  would  be  improved  when 
redundant  information  was  presented  in  both  modalities  at  once  (bimodal 
condition)  compared  to  the  situation  in  which  information  was  presented  to 
only  one  modality  to  which  attention  was  focused  (baseline  conditions),  or 
to  the  situation  in  which  information  was  presented  to  only  one  modality  tut 
the  subject  was  forced  to  divide  his  attention  between  two  modalities 
(unimodal  conditions).  Further,  it  was  hypothesized  that  detection  would  be 
degraded  when  the  subject  was  presented  with  information  in  one  modality  but 
required  to  attend  to  both  modalities  (unimodal  conditions)  compared  to  the 
situation  in  which  he  could  focus  on  one  modality  (baseline  conditions). 


METHOD 


Subjects 

Twenty-three  men  ages  18  to  24  years  volunteered  to  participate.  All 
had  or  were  corrected  to  20/20  visual  acuity  and  displayed  hearing  within 
the  normal  range  in  routine  audiometric  testing. 

Apparatus 

Visual  and  auditory  signals  were  initiated  by  a  Wavetek  programmable 
synthesized  function  generator  (model  278)  and  displayed  via  a  monochromatic 
visual  display  unit  (VDU)  and  Koss  (pro  4-AAA)  headphones.  The  generated 
signal  was  split  into  two  channels  and  fed  through  separate  attenuators 
prior  to  display.  The  noise  source  consisted  of  pre-recorded  ambient  sea 
noise  played  on  a  Hewlett-Packard  (3964a)  instrumentation  recorder.  The 
noise  signal  also  was  split  into  two  channels  and  routed  through  separate 
attenuators  to  the  monitor  and  headphones. 

The  visual  display  provided  signal  frequency  along  the  x-axis  and  time 
along  the  y-axis.  Visual  noise  appeared  as  random  lighted  pixels  varying  in 
intensity.  Amplitude  of  the  signal  was  represented  along  the  z-axis  which 
controlled  the  intensity  of  each  pixel.  A  horizontal  line  of  pixels 
appeared  at  the  top  of  the  display  and  moved  in  s  "waterfall"  fashion  down 
the  screen  (16  lines  present  at  a  time),  such  that  each  line  was  visible  for 
6.2  seconds.  A  visual  target  was  presented  at  either  600  Hz,  on  the  left 
side  of  the  display,  or  1700  Hz,  on  the  right  side  of  the  display.  The 
target  appeared  as  an  intermittent  vertical  arrangement  of  dots  of  greater 
intensity  than  the  background  noise.  The  amplitude  of  the  visual  noise  was 
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60  dB.  The  amplitude  of  the  visual  signal  initially  was  attenuated  to  below 
threshold  level,  and  attenuation  was  decreased  until  detection  occurred. 
Visual  detection  was  measured  in  terms  of  dB  of  attenuation. 

The  amplitude  of  the  auditory  noise  was  60  dB.  Targets  were  either  a 
600  Hz  or  1700  Hz  frequency  signal  providing  a  low  or  high  intermittent 
tone.  Targets  were  presented  as  tone  bursts  triggered  at  a  2  Hz  rate  with  a 
2  ms  pulse  width.  The  auditory  signal  also  was  attenuated  well  below 
threshold.  Attenuation  was  decreased  until  detection  occurred,  as  measured 
by  dB  level. 

Procedure 


Target  thresholds  were  measured  under  three  basic  conditions.  In  the 
baseline  condition,  only  one  sensory  modality  was  stimulated;  target 
information  and  noise  were  presented  either  as  a  visual  display  or  through 
earphones  and  attention  was  focused  on  the  presented  modality.  In  the 
unimodal  condition,  the  noise  was  presented  to  both  attended  modalities,  but 
the  target  information  was  presented  in  only  one  of  the  modalities.  In  the 
bjmodal  condition,  the  same  target  information  and  noise  were  presented  to 
both  attended  modalities  at  the  same  time. 

Subjects  were  tested  under  five  conditions:  (1)  auditory  target  with 
auditory  noise  (auditory  baseline);  (2)  visual  target  with  visual  noise 
(visual  baseline);  (3)  auditory  target  with  both  auditory  and  visual  noise 
(auditory  unimodal) ;  (4)  visual  target  with  both  visual  and  auditory  noise 
(visual  unimodal) ;  (5)  both  visual  and  auditory  target  with  both  visual  and 
auditory  noise  (auditory  and  visual  bimodal). 

After  a  brief  description  of  the  procedures  and  a  training  procedure, 
the  subjects  received  two  practice  trials  in  each  condition  with  feedback  as 
to  the  correctness  of  their  responses.  The  baseline  conditions  were 
presented  first  to  all  subjects  as  blocks  of  trials  in  counterbalanced 
order.  The  unimodal  and  bimodal  conditions  were  intermixed  and  presented 
randomly.  The  subjects  were  not  told  which  condition  was  being  presented. 
There  were.  30  trials,  six  per  condition.  Each  trial  took  about  one  minute 
to  complete. 

Targets  were  presented  below  threshold,  and  their  attenuation  was 
decreased  by  1  dB  every  6,2  seconds  (that  is,  their  intensity  was  increased) 
until  the  subject  reported  hearing  a  low  or  high  tone  and/or  seeing  a 
vertical  line  on  the  left  or  right  of  the  VDU.  In  the  bimodal  condition, 
attenuation  was  decreased  simultaneously  for  both  the  visual  and  auditory 
presentation  until  the  subject  correctly  reported  the  target  in  both 
modalities. 

Both  the  dB  level  of  attenuation  as  well  as  the  classification 
accuracy  was  recorded  for  each  target  detection.  The  classification  data 
were  used  to  eliminate  subjects  who  were  guessing  and  to  determine  if 
modality  preferences  existed  in  the  bimodal  condition. 
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RESULTS 


It  was  decided  to  eliminate  any  subjects  whose  classification  error 
rate  exceeded  25%.  Three  of  the  23  subjects  were  therefore  eliminated,  and 
their  data  were  excluded  from  the  analysis.  The  average  error  rate  of  the 
remaining  subjecs  was  11%. 

Levels  of  attenuation  (dB)  across  targets  were  converted  to  means  for 
each  of  the  five  conditions.  Difference  scores  were  determined  by 
subtracting  the  mean  attenuation  of  the  target  at  threshold  when  only  one 
modality  was  presented  and  attention  was  focused  (baseline)  from  the  mean 
attenuation  at  threshold  when  both  modalities  were  presented  and  attention 
was  divided.  Table  I  shows  these  difference  scores. 


Table  I.  Mean  Difference  Scores  Of  Unimodal  and  Bimodal 
Conditions  From  Baseline  ( dB  of  attenuation) 


Conditions 


Modality  Unimodal  Bimodal  Advantage 

Auditory  -.184  .942  1.126  ** 

Visual  -.504  1.123  1.627  ** 


**  j>  .01 

When  subjects  were  monitoring  two  modalities  and  were  presented  an 
auditory  target,  their  performance  was  similar  to  baseline.  They  were  a 
little  less  sensitive  to  the  target  (-.184  dB)  in  this  condition,  but  the 
difference  was  not  significant.  Similarly,  the  mean  attenuation  at 
threshold  of  the  visual  signal  when  subjects  monitored  two  modalities  was 
not  significantly  different  from  baseline  (-.504  dB).  In  other  words,  the 
change  from  focused  attention  to  divided  attention  when  only  one  target  was 
presented  did  not  significantly  degrade  detection  performance. 

When  the  same  target  was  presented  in  both  modalities  (bimodal),  Table 
I  shows  that  the  mean  attentuation  at  threshold  of  the  auditory  target  was 
.942  dB  greater  than  when  it  was  presented  only  as  an  auditory  signal.  That 
is,  the  target  was  detected  at  a  lower  threshold  when  it  was  accompanied  by 
the  presentation  of  the  visual  display.  This  increase  in  detectability  when 
both  targets  were  presented,  compared  to  the  condition  in  which  only  the 
auditory  signal  was  presented  and  there  was  no  visual  signal  in  the  visual 
noise,  was  highly  significant  (_F  (1, 19 )— 18.66,  ^<.01). 

Similarly,  in  that  condition  the  attenuation  of  the  visual  signal  was 
1.123  dB  greater  than  when  it  was  presented  only  as  a  visual  signal.  Again, 
the  visual  signal  was  detected  at  a  much  lower  threshold  when  it  was 
accompanied  by  the  auditory  presentation.  The  increase  in  detectability 
when  both  targets  were  presented  (bimodal)  compared  to  the  condition  in 
which  only  the  visual  signal  was  presented  and  there  was  no  auditory  signal 
in  the  auditory  noise  (unimodal)  was  highly  significant  (_F  (1,19)=18.66,  d< 
.01). 
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Auditory  detection  was  better  for  15  of  the  20  subjects,  and  visual 
detection  was  better  for  17  of  the  20  subjects  in  the  bimodal  condition, 
when  two  redundant  targets  were  presented  at  the  same  time  than  when  only 
one  target  was  presented. 


DISCUSSION 

The  results  of  this  study  support  the  major  hypothesis  that  bimodal 
presentation  of  a  redundant  target  reduces  detection  threshold  over  single 
target,  unimodal  presentations.  Detection  of  both  auditory  and  visual 
targets  was  enhanced  in  the  bimodal  condition,  regardless  of  the  modality  in 
which  the  target  was  first  perceived  by  the  subjects.  These  findings  are 
similar  in  nature  to  those  reported  in  vigilance  studies*®-^  and 
detection  studies  which  employed  redundant,  meaningful  information.^*^ 

Since  both  modalities  showed  an  enhancement  of  detection  performance 
when  the  stimuli  were  presented  simultaneously,  we  cannot  conclude  that  one 
stimulus  acts  as  an  accessory  for  another.  Rather,  the  integration  of  two 
redundant  signals  in  the  bimodal  condition  prior  to  a  response  seemed  to 
increase  the  subject's  sensitivity  and  certainty  that  a  particular  target 
had  been  presented.  Thus,  at  a  given  intensity,  two  redundant  signals 
presented  to  different  modalities  reduced  response  uncertainty  and  led  to 
more  efficient  performance  than  when  a  target  was  presented  to  a  single 
modality.  Adams  and  Chambers, 20  Corcoran  and  Weening, ^  and 
Duncan"^  formulated  similar  interpretations  based  on  their  experiments. 

The  hypothesis  that  monitoring  an  additional  modality  containing 
background  noise  would  negatively  affect  detection  of  a  single  target  was 
not  supported  statistically.  Subjects  were  able  to  move  from  focused 
attention  on  one  modality  to  divided  attention  across  two  modalities,  both 
with  background  noise,  without  a  significant  decrement  in  performance.  This 
finding  is  consistent  with  the  work  of  Mulligan  and  Shaw,  ^  and  suggests 
that  increasing  the  number  of  modalities  to  be  monitored  is  not  the  same  as 
increasing  the  set  size  of  target  information,  at  least  in  regard  to  two 
modalities. 

In  this  study,  detection  performance  on  a  simulated  sonar  task  was  not 
negatively  affected  when  a  second  modality  containing  background  noise  was 
introduced  and  attention  was  divided.  In  fact,  performance  increased  for 
both  auditory  and  visual  detection  when  redundant  targets  were  presented  in 
each  modality  simultaneously.  Both  of  these  findings  support  a  bimodal 
approach  to  sonar  operation  and  similar  types  of  signal  processing  tasks. 
Whether  these  results  would  replicate  using  other  dependent  measures  (i.e., 
reaction  time),  different  types  of  stimuli,  and  under  varying  task  loading 
or  time  sharing  conditions  will  need  to  be  explored,  particularly  in  regard 
to  such  operations  as  sonar,  radar,  and  air  traffic  control. 
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